About

Saroj Kumar Mishra is an experienced Big Data Architect with over 12 years of experience in designing and developing big data solutions using AWS and Azure cloud data managed services and open source technologies. He is currently working as a Staff Data Engineer at Tivo, where he has developed and designed the self-serve request-driven data extraction product for television viewership and advertisement data. Saroj has expertise in solving scalability issues in data pipeline and has hands-on experience designing solutions using AWS S3, EMR, Presto, Qubole data services, Airflow, Athena, SQS, and Python. He also has hands-on experience on Azure data lake, data factory, hdinsight, logic app, Blob store, and Azure sql DWH. Saroj has experience in designing and developing modules for data retention, quality, and processing. He is proficient in BI for structured and unstructured data modeling, data loading, data preparation, and data visualization. Saroj has designed and developed a platform to automate the spark and Hadoop ETL jobs using Airflow, Oozie, and Controlm. He has experience with large-scale distributed systems development, with expertise in handling the ETL process requirements for big data. Saroj is currently working as a Staff Engineer, Data Analytics R&D at Xperi Corporation, where he has been awarded for performance improvement and new technology and architecture adoption for TV data advertisement measurement and viewership data products ETL jobs. He has reduced operating costs by 80% and significantly reduced processing time, increasing capacity, higher scalability, and less monitoring. Saroj has also developed and designed the API-driven data extraction product for TV advertisement data measurement and modified the batch base ETL to event-driven architecture using AWS services like SQS, Lambda, Airflow, and EMR. In his past roles, Saroj has worked as a Lead Data Engineer at Maersk, where he helped the team in designing and building the next-generation data lake platform using Azure tech stack like Azure Data Factory ADF, Azure Data Lake Store ADLS, logic app, event hub, SQL dwh, and open-source big data technologies Spark, Scala, Hive, Avro, Parquet, and Hadoop. He also worked as a Senior Big Data Consultant at Emirates, where he built the data pipeline to process complex nested JSON and XML event data from booking, lounge, ticket, DCS source systems using Kafka and Flume integration. Saro